Sensing Capacity for Markov Random Fields

This paper computes the sensing capacity of a sensor network, with sensors of limited range, sensing a two-dimensional Markov random field, by modeling the sensing operation as an encoder. Sensor observations are dependent across sensors, and the sensor network output across different states of the environment is neither identically nor independently distributed. Using a random coding argument, based on the theory of types, we prove a lower bound on the sensing capacity of the network, which characterizes the ability of the sensor network to distinguish among environments with Markov structure, to within a desired accuracy.Comment: To appear in the proceedings of the 2005 IEEE International Symposium on Information Theory, Adelaide, Australia, September 4-9, 200

The Sensing Capacity of Sensor Networks

This paper demonstrates fundamental limits of sensor networks for detection problems where the number of hypotheses is exponentially large. Such problems characterize many important applications including detection and classification of targets in a geographical area using a network of sensors, and detecting complex substances with a chemical sensor array. We refer to such applications as largescale detection problems. Using the insight that these problems share fundamental similarities with the problem of communicating over a noisy channel, we define a quantity called the sensing capacity and lower bound it for a number of sensor network models. The sensing capacity expression differs significantly from the channel capacity due to the fact that a fixed sensor configuration encodes all states of the environment. As a result, codewords are dependent and non-identically distributed. The sensing capacity provides a bound on the minimal number of sensors required to detect the state of an environment to within a desired accuracy. The results differ significantly from classical detection theory, and provide an ntriguing connection between sensor networks and communications. In addition, we discuss the insight that sensing capacity provides for the problem of sensor selection.Comment: Submitted to IEEE Transactions on Information Theory, November 200

Compressive Sensing with Local Geometric Features

We propose a framework for compressive sensing of images with local distinguishable objects, such as stars, and apply it to solve a problem in celestial navigation. Specifically, let x be an N-pixel real-valued image, consisting of a small number of local distinguishable objects plus noise. Our goal is to design an m-by-N measurement matrix A with m << N, such that we can recover an approximation to x from the measurements Ax. We construct a matrix A and recovery algorithm with the following properties: (i) if there are k objects, the number of measurements m is O((k log N)/(log k)), undercutting the best known bound of O(k log(N/k)) (ii) the matrix A is very sparse, which is important for hardware implementations of compressive sensing algorithms, and (iii) the recovery algorithm is empirically fast and runs in time polynomial in k and log(N). We also present a comprehensive study of the application of our algorithm to attitude determination, or finding one's orientation in space. Spacecraft typically use cameras to acquire an image of the sky, and then identify stars in the image to compute their orientation. Taking pictures is very expensive for small spacecraft, since camera sensors use a lot of power. Our algorithm optically compresses the image before it reaches the camera's array of pixels, reducing the number of sensors that are required

Shift-encoded optically multiplexed imaging

In a multiplexed image, multiple fields-of-view (FoVs) are superimposed onto a common focal plane. The attendant gain in sensor FoV provides a new degree of freedom in the design of an imaging system, allowing for performance tradeoffs not available in traditional optical designs. We explore design choices relating to a shift-encoded optically multiplexed imaging system and discuss their performance implications. Unlike in a traditional imaging system, a single multiplexed image has a fundamental ambiguity regarding the location of objects in the image. We present a system that can shift each FoV independently to break this ambiguity and compare it to other potential disambiguation techniques. We then discuss the optical, mechanical, and encoding design choices of a shift-encoding midwave infrared imaging system that multiplexes six 15×15  deg FoVs onto a single one megapixel focal plane. Using this sensor, we demonstrate a computationally demultiplexed wide FoV video.United States. Air Force Office of Scientific Research (FA8721-05-C-0002

On the interdependence of sensing and estimation complexity in sensor networks

Computing the exact maximum likelihood or maximum a posteriori estimate of the environment is computationally expensive in many practical distributed sensing settings. We argue that this computational difficulty can be overcome by increasing the number of sensor measurements. Based on our work on the connection between error correcting codes and sensor networks, we propose a new algorithm which extends the idea of sequential decoding used to decode convolutional codes to estimation in a sensor network. In a simulated distributed sensing application, this algorithm provides accurate estimates at a modest computational cost given a sufficient number of sensor measurements. Above a certain number of sensor measurements this algorithm exhibits a sharp transition in the number of steps it requires in order to converge, leading to the potentially counter-intuitive observation that the computational burden of estimation can be reduced by taking additional sensor measurements

On the interdependence of sensing, accuracy, and complexity in large -scale detection applications

This thesis presents fundamental limits and practical algorithms for multi-sensor systems in 'large-scale detection' applications. Multi-sensor systems combine multiple sensor observations to detect or estimate the state of an environment. We define 'large-scale detection' as a detection problem where sensors are used to detect the state of a discrete grid or vector. Such problems characterize many important applications, including detection and classification of targets in a geographical area using a sensor network, and detecting complex chemicals using a chemical sensor array. While multi-sensor systems have been deployed in large-scale detection applications, many basic theoretical questions regarding sensor allocation and sensor selection have not been addressed in earlier work. To understand the performance limits of such systems, recent work proposed the idea of a 'sensing capacity.' While a definition was provided, the existence of a strictly positive sensing capacity, and therefore the practical value of this idea, remained an open question. One of the main contributions of this thesis is to define a sensing capacity that allows for a tolerable detection error. For this definition, we prove the existence of a positive sensing capacity for a number of multi-sensor system models. The sensing capacity provides fundamental limits on the number of sensor measurements required to detect the state of an environment. We obtain our theoretical results by using the insight that large-scale detection problems bear a striking resemblance to the problem of communicating over a noisy channel. In communications, Shannon's celebrated channel capacity results bound the maximum rate of transmission below which coding schemes with error probability arbitrarily close to zero are feasible. For a large-scale detection application, our results bound the minimum number of sensors required to achieve a desired detection accuracy with arbitrarily small probability of error. We analyze the sensing capacity for several multi-sensor system models, and extend this analysis to account for spatial and temporal dependence in the environment being sensed. Sensing capacity differs significantly from channel capacity, since it is not a mutual information. This has practical implications for the problem of sensor selection. In addition, our results differ significantly from classical detection theory, where the probability of error approaches zero as the ratio of hypotheses to sensor measurements goes to zero. In contrast, we show that there exists a positive ratio of the size of the grid or vector being detected to the number of sensor measurements below which error can be made arbitrarily close to zero. An important implication of our theoretical results is the connection provided by the sensing capacity between multi-sensor systems and the large number of coding ideas used to build communications systems. To demonstrate the benefit of this insight, we extend the idea of sequential decoding from convolutional codes to multi-sensor systems. Sequential decoding is a low-complexity decoding heuristic for convolutional codes that works well at rates sufficiently below the channel capacity. In simulations of robot mapping, we show that this idea can be applied to sensor fusion in multi-sensor systems, as an alternative to complex algorithms such as the belief propagation algorithm. To demonstrate the impact of the ideas presented in this thesis in a practical application, we show how a simple IR thermometer can be used to detect multiple targets. In practice, such sensors are used to estimate the temperature of a specific object. To enable the use of such sensors in target detection, we develop a realistic physics-based sensor model that accounts for the interaction of multiple hot targets with the sensor. In simulation, we demonstrate that for a sufficient number of IR sensor measurements, sequential decoding algorithms have sharp empirical performance transitions, becoming both computationally efficient and accurate. Empirically, the sequential decoding algorithm achieves accurate decoding with bounded computations per target position given a sufficient number of sensor measurements. This result suggests the existence of a 'computational cut-off rate' at rates sufficiently below the sensing capacity, similar to the one that exists for channel codes. In addition, these results demonstrate a computationally efficient way to obtain accurate detection using noisy, low-resolution sensors. Other approaches such as belief propagation achieved inferior accuracy with a significantly higher computational burden. We validate the feasibility of our approach in a series of experiments using an actual IR temperature sensor. Our results point to the possibility of building a cheap alternative to expensive IR cameras using simple IR sensors

Learning to detect partially labeled people

Deployed vision systems often encounter image variations poorly represented in their training data. While observing their environment, such vision systems obtain unlabeled data that could be used to compensate for incomplete training. In order to exploit these relatively cheap and abundant unlabeled data we present a family of algorithms called λMEEM. Using these algorithms, we train an appearance-based people detection model. In contrast to approaches that rely on a large number of manually labeled training points, we use a partially labeled data set to capture appearance variation. One can both avoid the tedium of additional manual labeling and obtain improved detection performance by augmenting a labeled training set with unlabeled data. Further, enlarging the original training set with new unlabeled points enables the update of detection models after deployment without human intervention. To support these claims we show people detection results, and compare our performance to a purely generative Expectation Maximization-based approach to learning over partially labeled data. 1